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SELECTING  AND  RANKING  COST  RESEARCH  PROJECTS 
(New/Innovative  Techniques  in  Parametric  Cost  Estimating) 

Dr.  William  H.  Jago 
Dr.  Lewis  S.  Fichter 

The  implementation  of  the  Strategic  Defense  Initiative,  now  called  Ballistic  Missile 
Defense  (BMD),  requires  developing  capabilities  to  protect  U.S.  forward  deployed  forces,  power 
projection  forces,  friends  and  allies,  and  the  United  States  against  ballistic  missile  attacks.  Such 
protection  now  primarily  involves  ground-based  systems  and  elements  to  ensure  continuous  global 
detection,  track,  and  intercept  of  ballistic  missiles  and  their  associated  warheads.  U.S.  Army 
efforts  include  coordinating  with  the  Ballistic  Missile  Defense  Organization  (BMDO);  conducting 
research,  development,  and  acquisition  of  strategic  and  tactical  systems  to  include:  Theater  and 
National  Missile  Defense  elements;  and  developing  appropriate  technology  bases.  The  role  of 
research  and  development  of  the  strategic  systems  is  the  responsibility  of  the  U.S.  Army  Space 
and  Strategic  Defense  Command  (USASSDC).  Within  USASSDC,  the  responsibility  for 
developing  cost  methodologies,  models,  databases,  and  Independent  Cost  Estimates  (ICEs) 
belongs  to  the  Cost  Analysis  Office  (CAO).  This  paper  describes  a  systematic  methodology  to  be 
used  to  select  and  rank  projects  in  support  of  the  CAO  cost  research  mission. 

The  paper  is  in  two  parts.  Part  1  identifies  a  large  list  of  candidate  cost  research  projects, 
and  their  descriptions,  which  are  divided  into  four  categories:  (1)  Databases;  (2)  Cost  Estimating 
Relationships  (CERs)  and  Methods;  (3)  Models;  and  (4)  Special  Studies.  Part  2  of  the  paper 
describes  in  detail  a  process  for  selecting  the  "best"  set  of  projects  within  limited  funding 
constraints  using  the  Analytic  Hierarchy  Process  (AHP)  in  conjunction  with  an  Integer  Linear 
Program  (ILP).  The  process  entails  three  major  steps:  (1)  developing  and  evaluating  a  multi¬ 
criteria  hierarchy;  (2)  determining  the  relative  value  of  each  potential  project  using  the  AHP 
method,  where  the  relative  values  or  "priorities"  are  based  on  factors  such  as  urgency, 
affordability,  payoff,  and  feasibility;  and  (3)  selecting  the  "best"  projects  using  ILP. 

The  decision  process  presented  in  this  paper  is  a  unique  application  of  operations  research 
decision  theory  which  could  be  tailored  to  many  decision-ms^ng  tradeoffs  that  the  DoD  faces.  A 
spin-off  of  this  effort  could  be  a  useful  tool  for  program  management  problems  that  transcend  cost 
research  planning. 

Dr.  Lewis  S.  Fichter 
Tecolote  Research  Corporation 
4950  Corporate  Drive,  Suite  140-0 
Huntsville,  Alabama  35805-6227 
(205)  895-0373 
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INTRODUCTION 


The  Anny  Strategic  Defense  Pro^am  is  the  implementation  of  the  Strategic 
Defense  Initiative  (SDI)  for  the  purposes  of  developing  ciq;>abilities  to  protect  U.S.  forward 
deployed  forces,  power  projection  forces,  friends  and  allies,  and  the  United  States  against 
ballistic  missile  attacks.  Such  protection  involves  surface  and  space  based  systems  and 
elements  to  enstire  continuous  global  detection,  track,  and  intercept  of  ballistic  missiles  and 
their  associated  warheads.  U.S.  Army  efforts  include  coordinating  with  the  Strategic  Defense 
Initiative  Organization  (SDIO);  conducting  research,  development,  and  acquisition  of 
strategic  and  tactical  systems  to  include  Theater,  National,  and  Global  Missile  Defense 
elements;  and  developing  appropriate  technology  bases.  The  role  of  research,  development, 
and  acquisition  of  these  systems  is  the  responsibility  of  the  U.S.  Army  Space  and  Strategic 
Defense  Command  (USASSDC).  Within  USASSDC,  the  responsibility  for  developing 
methodology,  models,  databases,  and  validating  cost  estimates  of  these  systems  belongs  to 
the  Cost  Analysis  Office  (CAO). 

CAO  is  involved  in  cost  estimating  in  all  phases  of  the  SDI  architectures  -  Global 
Protection  Against  Limited  Strikes  (GPALS),  Army  National  Missile  Defense  (ANMD),  and 
Army  Theater  Missile  Defense  (ATMD).  Figure  1-1  illustrates  the  integrated  system  concept 
of  GPALS. 
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Figure  I-l  Integrated  System  Concept  *  GPALS 


The  systems  associated  with  GPALS,  NMD,  and  TMD  can  be  further  subdivided 
into  near-term  and  far-term  systems  as  shown  in  the  Table  1-1 ; 


Table  1-1 

Strategic  and  Theater  Systems 


Systems 

Stral 

tegic 

Theater 

Near-Term 

Far-Term 

Near-Term 

Far-Term 

GBR  -  NMD 

X 

GSTS 

X 

GBI 

X 

CCE 

X 

ERINT 

X 

PATRIOT  P3I 

X 

ARROW/ACES 

X 

THAAD 

X 

CORPS  SAM 

X 

GBR  -  TMD 

X 

ATOC 

X 

Free  Election  Laser  (FEL) 

X 

Neutral  Particle  Beam  (NPB) 

X 

Electrothermal  Chemical  Gun 

X 

Laser  Radar 

X 

The  technology  development  program  associated  with  the  above  systems  can  be 
subdivided  between  the  necessary  subsystem  areas  associated  with  sensors,  kinetic  and 
di'  ccted  energy  weapons,  and  BM/C3,  Table  1-2  shows  this  mapping: 
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Table  1-2 

USASSDC  Technology  Areas 


Technologies 

USASSDC  System  Areas 

Sensors 

Kinetic 

Energy 

Weapons 

(KEW) 

Directed 

Energy 

Weapons 

(DEW) 

BMC3 

Advanced  Materials/Materials  Processing 

X 

X 

Advanced  Signal  Processing/Computing 

X 

X 

X 

X 

Directed  Energy 

X 

Low  Observables  Technology 

X 

Microelectronics/Photonics 

X 

X 

Advanced  Propulsion  Technologies 

X 

Space  Technology 

X 

X 

X 

Advanced  Manufacturing  Technologies 

X 

X 

X 

X 

The  remainder  of  this  report  is  contained  in  two  sections.  Section  2  presents  the 
recommended  projects  and  their  description,  Section  3  discusses  the  methodology  used  to 
select  and  rank  the  chosen  projects.  Section  4  presents  a  schedule  of  projects  selected  from 
Section  2  by  applying  the  methodology  described  in  Section  3. 
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2 

roENTIFICATION  OF  REQUIREMENTS 


This  section  presents  the  candidate  projects  and  a  brief  description  of  each. 

2.1  CANDIDATEJ>&QJECIS 

Candidate  projects  were  divided  into  four  basic  categories:  (1)  databases,  (2)  Cost 
Estimating  Relationship  (CERs)  and  methods,  (3)  major  computer  models,  and  (4)  special 
studies.  The  selection  of  candidate  projects  was  done  by  a  combination  of  interview  with 
experts  within  Tecolote  and  by  reviewing  any  pertinent  literature,  such  as  the  Army 
Technology  Base  Master  Plan^.  Please  note  that  projects,  programs,  and/or  technologies 
managed  by  USASSDC  that  might  be  expected  to  be  listed  may  simply  be  excluded  because 
of  recent  cost  research  efforts;  e.g..  Extended  Range  Interceptor  (ERJNT),  Airborne  Optical 
Adjunct  (AOA),  Airborne  Surveillance  Testbed  (AST),  and  PATRIOT  PAC  II  which  are 
currently  in  process  as  factbooks.  The  candidate  projects  selected  are  as  follows: 

Databases 

Fact  Books 

High  Endoatmospheric  Interceptor  (HEDI)  Subsystem 
Exo-Endoatmospheric  Interceptor  (E2I) 

Ground  Surveillance  and  Tracking  Subsystem  (GSTS) 

Exoatmospheric  Reentry  Vehicle  Interceptor  Subsystem  (ERIS) 
Ground  Based  Interceptor  (GBI) 

Aegis 

Cheyerme  Mountain 

Anti-Satellite  (ASAT)  System 

Theater  High  Altitude  Area  Defense  (THAAD) 

Ground  Based  Radar  (GBR) 

Technology  Programs 

Light  Exoatmospheric  Projectile  (LEAP) 

Cost  Estimating  Handbook 
Ada  Software 


*Annv  Technology  Ba.se  Master  Plan.  Volumes  1  and  11,  Febniary  1992. 


5 


Cost  Estimating  Relationships  (CERs)  and  Methods 
Infrared  Focal  Plane  Array  Seekers 
Telescope 
Signal  Processor 
Detection  Arrays 
Dewar/Cryostat 
Window  Cooling 
Radome  Materials 
Visible  Seekers 
Advanced  Solid  Propellants 

Battle  Management/Conunand,  Control  and  Communications  (BM/C3) 
Command  and  Control  Element  (CCE) 

Ground  Entry  Point  (GEP) 

Engagement  planning  (EP) 

Hypervelocity  Gun  (HVG)  Technology 
Directed  Energy  Weapons 

Free  Electron  Laser  (FEL) 

Chemical  Laser 

Neutral  Particle  Beam  (NPB) 

Pointing,  Tracking,  and  Retargeting 
Materials  Cost  Impacts 

Warheads/Lethality  Enhancement  Devices  (LED) 

Launchers 
Investment  Items 

Integration  of  Cost  and  Technical  Risk  Factors 

System  Engineering  and  Integration  (ST&E)  Function  Point  Analysis 
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Models 

Radar  Engineering  and  Cost  Tool  (REACT)  Enhancements 
System  Trades  Architecture  Cost  Model  (STACM)  Enhancements 
Automated  Cost  Estimating  Integrated  Tools  (ACEIT)  Libraries 
BM/C3  GEP  Engineering  and  Cost 
BM/C3  EP  Engineering  and  Cost 

BM/C3  Command  Control  Element  (CCE)  Engineering  and  Cost 
System  Test  and  Evaluation  (ST&E)  Life  Cycle  Cost  Trade-off  Model 

Special  Studies 

Software  Sizing  Risk  Analysis 

Selected  Acquisition  Report  (SAR)  Cost  Growth 

BM/C3  Sizing 

Microelectronics  Sizing 

Analog  Selection 

Software  Function  Point  Analysis 

Artificial  Intelligence  -  AI/Expert  Systems 

Linear  Discriminators 

2.2  PROJECT  DESCRIPTIONS 

This  section  includes  a  short  description  of  the  proposed  effort  that  would  be 
conducted  under  the  various  projects. 

2.2.1  DATABASES 

2.2. 1.1  Factbooks 

The  development  of  Factbooks  will  document  the  threat  requirement,  mission, 
heritage  evolution,  technical  contract  program,  and  cost  history  data.  This  would  provide 
"lessons  learned"  and  reference  information,  as  well  as  provide  additional  cost  data  for  use  in 
the  development  of  Cost  Estimating  Relationships  (CERs),  Cost-to-Cost  (CTC)  factors, 
analogs,  and  other  vital  cost  and  performance  information  to  enhance  future  projections. 
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HEDliE^i) 

The  HEDI  (E^I)  program  is  a  source  of  valuable  information.  Recently  canceled, 
the  information  may  be  unavailable  if  not  applied  soon.  Oin  subcontractor,  CRC,  is  well 
qualified  by  their  direct  involvement  to  efficiently  and  effectively  support  this  effort. 

GSTS 

Relevant  hardware  (especially  sensor)  and  software  data  would  be  provided  in  the 
areas  of  missile  components. 

ERISCGBI) 

This  program  is  the  GPALS/NMD  interceptor.  Our  subcontractor,  SPARTA,  has 
supported  GBI  for  years  and  would  be  a  valuable  resource  on  this  project.  It  should  include 
the  Homing  Overlay  Experiment  (HOE),  ERIS,  Ground  Based  Interceptor  Experimental 
(GBI-X),  and  GBI. 

AEGIS 

The  U.S.  Navy  AEGIS  Weapon  System  is  a  sophisticated  configuration  of 
integrated  hardware  and  software  subsystems.  (The  word  "aegis"  which  has  associations 
with  Greek  and  Roman  mythology,  means  "protection"  or  "defense".)  It  embodies  several 
sensors,  five  or  more  weapons,  and  BM/C^  for  air,  surface,  and  antisubmarine  warfare.  It  is  a 
relevant  analog  for  many  aspects  of  SDI. 

AEGIS  represents  the  single-most,  complex  system  integration  and  system  test  data 
point  prior  to  SDI.  As  such,  there  is  direct  application  potential  in  the  areas  of  System 
Engineering  and  Integration  (SE&I),  BM/C^I,  software,  testing  of  BM/C^I  and  complex 
integrated  strategic  weapon  systems,  and  many  other  technical  and  cost  components  of  a 
complex  weapon  system. 

Cheyenne  Mountain 

The  BM/C^  (C^E)  program  is,  in  part,  a  modification  to  the  Cheyenne  Mountain 
AFB  Command  Center.  There  are  currently  five  upgrades  in  process.  We  need  to  recognize 
and  understand  that  baseline  for  the  further  enhancements  to  accomplish  the  Strategic 
Defense  Initiative  (SDI). 
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ASAT 


Relevant  hardware  and  soitware  data  would  be  provided  especially  in  areas  of 
missile  components  (e.g.,  boosters  and  sensors). 

HiAAD 

Relevant  hardware  and  software  data  would  be  provided;  especially  in  the  areas  of 
missile  components,  TMD  sensors,  launcher,  and  Tactical  Operations  Center. 

Ground  Based  Radar  (GBRl 

A  reference  that  captures  the  evolution  of  USASSDC  radar  concepts  from  the 
Terminal  Imaging  Radar  (TIR)  and  GBR-X  up  to  the  current  GBR-TMD  and  GBR-NMD. 

Technology  Programs 

The  development  of  either  an  overall  Technology  Program  Faetbook  or  a  series  of 
specific  Technology  Faetbooks  would  be  useful  to  document  the  technical,  programmatic, 
schedule,  research  efforts,  contract,  and  cost  data  for  the  various  Kinetic  Energy  Weapons 
(KEW),  Directed  Energy  Weapons  (DEW),  Survivability  Lethality  and  Kill  Technology 
(SLKT),  and  component  technology  programs.  This  would  provide  useful  analog  and 
potential  CER  cost  data  for  use  in  future  cost  and  budget  estimates.  In  addition,  these 
document(s)  would  provide  data  for  the  CARD  development  and  information  on  the  potential 
impact  of  "leveraging"  from  the  technology  programs. 

LEAP 

The  development  of  the  Light  Exoatmospheric  Projectile  (LEAP)  Faetbook  would 
provide  information  regarding  the  Army  LEAP  and  endo-leap  programs  on  technical 
programmatic,  schedule,  research  activities,  tests,  contracts,  and  cost  data.  This  information 
would  be  useful  in  the  development  of  small/miniature  components  and  projectile 
capabilities.  Information  regarding  the  Air  Force  LEAP  program  might  also  be  included. 

2.2. 1.2  Cost  Estimating  Handbook 

The  development  of  a  Cost  Estimating  Handbook  would  give  a  ready  CER,  factor 
and  analog  reference  for  cost  analysts.  It  would  be  organized  pursuant  to  the  US  Army  "Big 
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Six"  or  "Big  Three"  cost  element  structures  (CES)  for  all  life  cycle  phases.  Other  cost 
documents  would  be  used  in  addition  to  cost  databases,  cost  research  reports,  and  other 
literature  to  provide  the  various  CERs,  factors,  and  analogs.  The  initial  construction  would 
be  heavily  dependent  on  existing  cost  research;  while  in  the  future,  original  research  might  be 
conducted  to  enhance  the  handbook.  This  handbook  might  be  similar  in  context  to  the 
Unmanned  Spacecraft  Cost  Model  (USCM)  series  or  the  Air  Force  Systems  Command  Cost 
Estimating  Handbooks. 

2.2.1. 3  Ada  Software 

Develop  a  database  of  Ada  software  projects  of  interest  to  USASSDC.  This  would 
include  cost  and  technical  parameters  such  as  size,  effort,  complexity,  and  others. 

2.2.2  CERs  AND  METHODS 

2.2.2.1  Infrared.  Focal  Plane  Array  Seekers 

This  technology  is  being  implemented  in  the  GBI  and  THAAD  interceptors.  There 
are  many  programs  in  all  three  Services  that  are  pursuing  "second  generation"  focal  planes. 
Data  collection  would  be  for  methodology  improvement  and  surveys  of  other  estimating 
techniques  for  crosscheck  capability.  The  components  include  telescope,  signal  processor, 
detection  arrays,  dewar/cryostat,  window  cooling,  and  radome  materials. 

2.2.2.2  Visible  Seekers 

Many  USASSDC  interceptor  systems  are  characterized  by  visible  seekers  to 
supplement  the  target  acquisition  function  performed  by  IR  seekers.  This  research  task  will 
identify  and  collect  data  on  current  visible  seeker  and  sensor  systems  and  to  develop  CERs. 

2.2.2.3  Advanced  Solid  Propulsion 

Current  solid  rocket  motor  technology  is  well  characterized  by  current  CERs. 
However,  advanced  features  such  as  insensitive  munitions  and  gels  need  to  be  researched. 

2.2.2.4  BM/c3  CCE 

A  cost  estimating  reference  for  the  Command  and  Control  Element  of  strategic 
defense  architectures.  This  will  include  analogous  data  and  other  relevant  cost  estimating 
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methodology.  Evolving  processor  technology  limits  the  '*shelf  life"  of  CERs  and  requires 
frequent  updates. 

2.2.2.5  BM/c3  Ground  Entry  Point 

A  cost  estimating  reference  for  Satellite  and  Interceptor  Ground  Entry  Points  (GEP). 
This  study  will  include  analogous  data,  CERs,  and  other  relevant  cost  estimating 
methodologies. 

2.2.2.6  BM/c3  Engagement  Planning 

A  cost  estimating  reference  for  Engagement  Plan  or  (Battle  Management)  systems. 
This  will  include  analogous  data,  CERs,  and  other  relevant  cost  estimating  methodologies. 

2.2.2.7  Hvpervelocitv  Gun  Technology 

Updating  of  the  study  from  Task  Assignment  91-003,  titled  CR-0542,  "Hyper 
Velocity  Gun  (HVG)  Technical  Report  and  Cost  Methodology",  November  1991,  would 
include  data  collection  and  an  update  of  the  CERs. 

2.2.2.8  Directed  Energy  Technologies 

The  development  of  Directed  Energy  CERs  and  methods  would  be  useful  in 
assisting  the  cost  analyst  in  the  development  of  leap  ahead  technologies  under  consideration 
in  strategic  and  TMD  scenarios.  Items  that  might  be  included  are  Free  Election  Laser  (FEL), 
chemical  laser,  Neutral  Particle  Beam  (NPB),  Pointing,  Tracking,  Retargeting,  etc.  Directed 
Energy  Weapons  (DEW)  will  continue  to  be  examined  and  possibly  be  the  weapons  of  the 
future.  Methodologies  need  to  be  improved  to  support  existing  estimating  requirements. 

2.2.2.9  Material  Cost  Impacts 

Task  91-007  identified  a  number  of  new  materials  and  potential  applications  to 
USASSDC  systems.  Within  the  task,  "IR  Windows  and  Structures"  were  pursued  for  CER 
development.  There  are  many  follow-on  candidates  from  that  list  (including  updating  the  list 
or  adding  data  to  update  the  Window  CER). 
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2.2.2.10 


The  development  of  estimating  CERs  and  methods  for  warheads  and  Lethalit>' 
Enhancement  Devices  (LEDs)  is  an  area  that  has  not  been  sufficiently  examined.  Very  few 
CERs  exist  and  most  estimates  are  of  an  analog  nature.  This  is  an  excellent  opportunity  to 
produce  a  comprehensive  warhead/LED  cost  estimation  document  for  use  throughout  DoD. 
The  TMD  environment  will  include  fiagmentation  type  warheads  in  addition  to  the  kinetic 
kill  LED  methods  and  should  be  examined. 

2.2.2.11  Launchers 

The  development  of  updated  launcher  CERs  and  data  would  be  very  timely  since 
many  of  the  CERs  date  back  to  1983.  The  requirements  of  various  TMD  systems  include 
"roll-on,  roll-off',  C-130  capability,  and  volume  and  weight  constraints.  The  Army  needs 
new  types  of  launchers,  lighter  materials  (e.g..  Boron  Epoxy),  smaller  platforms  (e.g.,  5  ton), 
stabilization,  blast  shields,  erection  systems,  and  data  interchange.^ 

2.2.2.12  Investment  Items 

This  effort  should  be  accomplished  to  develop  CERs,  factors,  and  analogs  for 
various  Investment/Production/Procurement  Items.  A  databook  and/or  collection  of 
investment  cost  element  methods  in  a  document  would  be  useful  for  quick  reference.  Data 
collection  and  literature  research  should  be  conducted  to  provide  a  comprehensive  listing. 
New  CERs  and  factors  could  be  developed  using  the  revised  and  stratified  databases. 

2.2.2.13  Integration  of  Risk  Factors 

The  general  methodologies  dealing  with  Risk  Analysis  are  well  defined  but 
typically  are  applied  in  a  one-dimensional  way  (i.e.,  cost,  technical,  or  schedule).  The 
interrelationships  between  cost  risk  and  technical  risk  are  intuitively  clear,  but  methodologies 
to  capture  the  total  risk  impact  (addressing  the  statistical  dei)endency  between  the  two)  are 
limited.  This  proposed  research  effort  would  develop  a  rigorous  applied  methodology  to 
derive  risk  factors  which  measure  the  effects  of  correlated  cost  and  technical  risk  postures. 


^e.g.,  Enhanced  Position  Location  Reporting  System  (EPLRS),  and  Joint  Tactical  Information  Distribution 
System  (JTIDS),  etc. 


2.2.2.14  SE&I  Function  Points 

Study  the  application  of  function  point  counting  to  the  problem  of  estimating  the 
cost  of  System  Engineering  and  Integration  (SE&I).  This  task  will  include  data  collection 
and  methodology  development. 

2.2.3  MODELS 

2.2.3. 1  Radar  Engineering  and  Cost  Tool  (REACTT  Enhancements 

This  is  a  useful  trade  study  model.  There  are  a  number  of  further  developments 
(input/outputs,  analysis  features,  testing,  and  documentation  improvements)  that  are 
recommended. 

2.2.3.2  STACM  Enhancements 

This  top-level  trade  study  model  is  currently  populated  with  TMD  systems.  NMD 
systems  can  be  added  to  the  templates  and  libraries.  Putting  the  model  into  operation  may 
identify  user  defined  improvements. 

2.2.3.3  ACEITLibrarigs 

A  library  of  USASSDC  CERs  and  factors  could  be  loaded  into  ACE  IT  for 
productivity  in  estimating  and  documentation.  This  could  also  include  the  methods  such  as 
those  described  in  2.2. 1.2  and  all  of  2.2.2. 

3.2.3.4  GEP  Engineering  &  Cost 

Development  of  a  GEP  Engineering  and  Cost  Model  would  be  useful  in 
Independent  Cost  Estimates  and  architecture  studies. 

2.2.3.5  ST&E  Life  Cycle  Cost  (LCCl  Trade-off  Model 

Building  upon  the  ST&E  Cost  Guide  proposed  for  the  FY93  efforts  would  be  the 
development  of  a  stand  alone  automated  ST&E  cost  and  trades  model.  The  model  might 
consist  of  some  of  the  features  of  the  STACM,  REACT,  and  other  automated  estimating 
models.  Ideally  it  would  have  an  ST&E  cost  library  and  database  attached  whereby  selection 
of  ranges,  targets,  and  launch  vehicles  could  be  conducted.  The  input  of  quantities  and  time- 
phased  schedules  would  produce  a  time-phased  estimate  in  constant  and  escalated  dollars. 
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An  excursion  and  optimization  capability  and  a  top-level  flow  chart  or  network 
(automatically  produced)  might  be  possible.  The  model  would  be  extremely  useful  to  project 
offices  and  test  organizations. 

2.23.6  Engagement  Planning  Engineering  &.  Cost  Model 

It  would  be  useful  to  develop  an  Engagement  Planning  (EP)  Engineering  and  Cost 
Model  for  Independent  Cost  Estimates  and  architecture  studies. 

2.23.7  CCE  Engineering  &  Cost 

The  development  of  a  Command  and  Control  Element  (C^E)  Engineering  and  Cost 
Model  would  be  useful  in  Independent  Cost  Estimates  and  architecture  studies. 

2.2.4  SPECIAL  STUDIES 

2.2.4.1  Software  Sizing  Risk  Analysis 

Develop  a  study  of  software  sizing  and  its  impact  on  software  cost  estimates. 
Develop  a  means  to  quantify  the  risk  in  software  sizing  estimates. 

2.2.4.2  SAR  Cost  Growth 

The  historical  track  of  cost  change  (variance)  for  major  Department  of  Defense 
(DoD)  programs  is  captured  in  the  Selected  Acquisition  Report  (SAR).  This  data,  when 
analyzed  in  the  context  of  the  SAR  variance  analysis  statements,  provides  an  excellent  source 
of  program  cost  change  over  a  relatively  long  and  continuous  time  frame.  This  proposed 
research  effort  would  develop  expected  cost  change  (growth)  facto:  for  future  programs 
based  on  Service,  Weapon  type,  system  complexity,  etc.  These  factors  can  be  used  for  cross¬ 
checking  the  more  rigorously  developed  risk  (TRACE/TRACE-P)  estimates. 

2.2.4.3  BMZC3.Sizmg 

The  data  collection  and  research  of  BM/C^  requirements,  performance,  technical, 
software,  staffing  and  other  interrelationships  could  be  useful  in  assessing  the  realism  of  the 
Cost  Analysis  Requirements  Document  (CARD)  baseline. 
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2.2.4.4 


The  development  of  a  study  of  microelectronic  sizing  methodologies  should  be 
performed  to  include  the  development  of  new  methods  for  sizing  the  quantity  and  type  of 
microelectronics  components. 

2.2.4.5  Analog  Selection 

Development  of  an  analytic  method  for  selecting  and  scaling  the  best  analog 
subsystem/component  to  the  desired  item,  based  on  that  item's  technical  and  performance 
characteristics.  This  methodology  could  be  used  when  there  is  not  enough  relevant  data  to 
apply  high  quality  CERs  and  for  cross  checks  of  other  estimating  methodologies. 

2.2.4.6  Software  Function  Point  Analysis 

Development  of  a  study  of  the  application  of  Software  Ftmction  Point  (FP)  Analysis 
techniques  to  USASSDC  projects.  Collect  and  analyze  FPs  on  selected  projects. 

2.2.4.7  AI/£xpert  Systems 

Investigate  generating  Al/expert  system  rules  from  numerical  data  using  fuzzy  logic 
calculus.  Recent  advances  in  Europe  and  Japan  applying  fuzzy  logic  calculus  to  engineering 
design  and  estimating  problems  indicate  that  these  techniques  might  be  useful  in  cost 
estimating  where  there  is  little  relevant  data.  Also,  obtain  and  evaluate  recent  commercial 
Al/expert  system  literature  and  packages  for  potential  application  to  the  USASSDC  data 
environment. 

2.2.4.8  Linear  Discriminators 

Discriminant  analysis  is  a  multivariate  statistical  technique  can  be  used  to  develop 
combinations  of  sets  of  variables,  which  are  criteria  for  group  differentiation.  A  linear 
combination  of  such  variables  is  called  a  "linear  discriminant  function".  This  proposed 
research  effort  would  develop  linear  discriminant  functions  to  be  used  for  classifying  or 
assigning  given  "products"  to  a  specific  group.  For  example,  a  discriminant  ftmction  could 
be  developed  to  classify  a  proposed  program  schedule  as  a  "high",  "nominal",  or  "low"  risk 
schedule.  Similarly,  functions  could  be  developed  for  assigning  or  classifying  a  cost  estimate 
within  a  range  of  "not  probable",  "probable",  or  "most  probable". 
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3 

METHODOLOGY 


3.1  DECISION  TOOL  CONCEPT 

The  process  of  selecting  the  "best”  set  of  projects  within  limited  funding  constraints 
can  be  a  complicated  resource  allocation  problem.  Unfortunately,  almost  every  real  world 
decision  involves  multiple  objectives.  Use  of  traditional  optimization  techniques,  such  as 
linear  programming  (LP),  are  limited  to  optimizing  a  single  objective  subject  to  a  number  of 
constraints.  Thus,  a  pure  LP  approach  to  the  resource  allocation  problem  often  appears  to  be 
"forced".  There  is  a  strong  likelihood  that  senior  management  will  (rightfully)  feel 
uncomfortable  with  the  analysis  and  not  make  proper  use  of  it  in  their  decisions.  Instead, 
they  will  tend  to  ponder  how  to  evaluate  the  tradeoffs  between  the  pros  and  cons  of  each 
potential  project  with  respect  to  each  of  their  "true"  objectives,  which  will  be  some  mixture 
of  quantitative  and  qualitative  considerations. 

The  Analytic  Hierarchy  Process^  (AHP)  is  a  method  that  formally  structures  multi¬ 
objective  tradeoffs.  Considering  the  pros  and  cons,  the  decision  maker  structures  the 
organization's  primary  objectives,  sub-objeciives,  and  so  on  into  a  hierarchy.  Using  the  AHP 
method  to  evaluate  the  hierarchy  produces  a  set  of  weighted  numerical  averages  which  reflect 
the  decision  maker's  relative  priorities.  Thus,  each  potential  project  has  a  numerical  measure 
that  represents  its  overall  value  with  respect  to  all  the  identified  selection  criteria  (qualitative 
as  well  as  quantitative).  Using  these  measures  as  the  basis  of  an  objective  function,  an 
Integer  Linear  Program  (ILP)  can  be  used  to  determine  the  "best"  selection  of  projects  to 
undertake  in  a  given  time  period,  subject  to  budgetary  and  other  constraints. 

The  remainder  of  this  section  describes  the  details  of  the  process  used  in  developing 
the  research  road  map  project  selection  for  USASSDC/CAO.  In  brief,  the  process  entails 
three  major  steps:  (1)  developing  and  evaluating  a  multi-criteria  hierarchy,  (2)  determining 
the  relative  value  of  each  potential  project  using  the  AHP  method,  and  (3)  selecting  the  "best" 
projects  using  ILP.  The  selection  of  potential  projects  is  discussed  in  Section  4. 


^Saaty,  T.L.,  The  Analytic  Hierarchy  Process.  McGraw-Hill  book  Co.,  1980. 
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In  providing  cost  estimating  and  validation  to  USASSDC,  the  CAO  depends  on 
research  to  develop  the  databases.  Cost  Estimating  Relationships  (CERs)  and  methods, 
detailed  computer  models,  as  well  as  special  studies  to  support  their  estimates.  In  choosing 
projects  for  these  research  areas,  CAO  needs  to  consider  each  project's  urgency,  affordability, 
payoff,  and  feasibility  of  successful  completion.  Figure  3-1  shows  the  AHP  hierarchy  built  to 
represent  these  primary  considerations.  In  addition,  pairwise  comparisons  were  made  to 
derive  the  relative  priorities  for  project  urgency,  payoff,  and  feasibility  by  using  qualitative 
rating  scales  varying  from  very  low  (VLO)  to  very  high  (VHI).  Affordability  was  rated  using 
a  set  of  expected  cost  ranges  (e.g.,  $50  -  lOOK).  The  number  below  each  name  in  Figure  3*1 
is  the  relative  priority^  of  that  item  to  the  other  items  at  the  same  level  in  the  hierarchy.  For 
example,  0.1102  is  the  relative  priority  of  databases  with  respect  to  CERs/methods,  models, 
and  special  studies.  Note  that  the  sum  of  the  values  at  the  same  level  in  the  hierarchy  are 
equal  to  one. 

3.3  PRQJECT.RELATIVE  VALUE 

The  relative  values  or  priorities  calculated  for  each  project  are  determined  by 
evaluating  the  project  at  the  urgency,  affordability,  payoff,  feasibility  level  of  the  hierarchy. 
For  example,  a  database  project  might  be  very  urgent,  cost  $75K,  have  a  nominal  payoff,  and 
have  a  high  feasibility  of  successful  completion.  In  words,  the  relative  value  computation  is 
as  follow: 

Relative  Value  =  Database  *  (Urgency  *  HI  +  Afford  *  (50  -  lOOR  Payoff  *  NOM  +  Feasible  *  HI) 

Referring  to  Figure  3-1,  Analytical  Hierarchy  Process  (AHP)  Prioritization  Schematic,  the 
above  word  formula  gives  the  following  numerical  value: 

Relative Value=0. 1 1 02»(0.2704*0.2635-K).32 14*0.01 79-K).2869*0.0764-K).  1 222*0.2635)=0.0 1444968 


^Mathematically,  the  priorities  are  the  normalized  eigenvectors  of  the  principal  eigenvalue  from  the 
reciprocal  matrix  of  ratio  scale  pairwise  comparisons. 


17 


Table  3-1  contains  the  relative  values  for  the  potential  projects  considered  in  this  evaluation. 
The  commercial  software  package  Expert  Choice  by  Decision  Support  Software,  Inc.  was 
used  to  perform  the  AHP  calculations. 
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PRIORITIZE  SDC  PROJECTS 


Figure  3-1  Analytical  Hierarchy  Process  ( AHP)  Prioritiration  Schematic 


TABLE  3-1 

PROJECT  RELATIVE  VALUES 


PROJECTS 


URGENCY 

AFFORD 

PAYOFF 

FEASIBLE 

RELATIVE 

0.02979808 

0.03S41828 

0.03I6I638 

0.01346644 

VALUE 

DATABASES 

Factbooks 


HEDl  (E21) 

VHl 

50-100K 

VHI 

VHI 

004369799 

GSTS 

LOW 

30-100K 

NOM 

HI 

0.00798648 

ERIS  (GBI) 

VHI 

50-100K 

VHI 

VHI 

0.04369799 

Aegis 

VHI 

SO-IOOK 

VHI 

VHI 

0.04369799 

Cheyenne  Mtn. 

NOM 

50-100K 

NOM 

VHI 

0.01307060 

ASAT 

VHI 

50-I00K 

VHI 

VHI 

0.04369799 

THAAD 

NOM 

SO-IOOK 

NOM 

VHI 

0.01307060 

GBR 

LOW 

50-100K 

NOM 

HI 

0.00798648 

Technology  Programs 

NOM 

50-100K 

HI 

HI 

0.01478988 

LEAP 

NOM 

50-100K 

NOM 

HI 

0.00887446 

Cost  Est.  Handbook 

HI 

I00-150K 

VHI 

VHI 

0.03887916 

ADA  S/W 

NOM 

50-100K 

HI 

HI 

0.01478988 

SCADS 

HI 

50-I00K 

HI 

HI 

0.02036510 

Hazard  Waste  Handbook  HI 

VHI 

50-1 OOK 

VHI 

VHI 

0.04369799 
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TABLE  3-1 

PROJECT  RELATIVE  VALUES  (Cont’d) 


URGENCY 

AFFORD 

PAYOFF 

FEASIBLE 

RELATIVE 

C£Rs/M£THODS 

0.016375424 

0.19463984 

0.17374664 

0.7400432 

VALUE 

IR  FPA  SEEKERS 


TELESCOPE 

HI 

<50K 

NOM 

HI 

0.08468242 

SIGNAL  PROCESSOR 

VHI 

50-100K 

NOM 

HI 

0.13043350 

DETECTION  ARRAYS 

VHl 

50-100K 

NOM 

HI 

0.13043350 

DEWAR/CRYOSTAT 

HI 

<50K 

NOM 

HI 

0.08468242 

WINDOW  COOLING 

HI 

<50K 

HI 

NOM 

0.13034420 

RADOME  MATERIALS 

NOM 

<50K 

NOM 

NOM 

0.14019779 

VISIBLE  SEEKERS 

VHl 

50-100K 

NOM 

HI 

0.13043350 

ADV.  SOLID  PROP 

HI 

50-100K 

VHl 

HI 

0.16605513 

BM/C3  CCE 

LOW 

50-100K 

LOW 

HI 

0.03871173 

BM/C3  GEP 

LOW 

50-100K 

LOW 

HI 

0.03871173 

BM/C3  EP 

LOW 

50-100K 

LOW 

HI 

0.03871173 

HV  GUN  TECHNOLOGY 
DIRECTED  ENERGY 

NOM 

<50K 

NOM 

LOW 

0.03799246 

FREE  ELECTRON  LASER 

LOW 

<50K 

LOW 

LOW 

0.02793494 

CHEMICAL  LASER 

LOW 

<50K 

LOW 

LOW 

0.02793494 

NEUTRAL  PART.  BEAM 

VLO 

<50K 

VLO 

LOW 

0.02516743 

POINT.  TRACK,  RETARG 

NOM 

<50K 

NOM 

LOW 

0.03799246 

MATRL  COST  IMPACTS 

HI 

<50K 

HI 

NOM 

0.10334420 

WARHEADSLEDS 

HI 

<50K 

HI 

NOM 

0.10334420 

LAUNCHERS 

NOM 

50-100K 

HI 

HI 

0.08127726 

INVESTMENT  ITEMS 

HI 

50-100K 

HI 

VHl 

0.13497542 

INTEGR.  RISK  FACTORS 

HI 

<50K 

HI 

\N1 

0.14025016 

SE&l  FUNCTN  POINTS 
BELOW  THE  LINE  COSTS 

NOM 

50-100K 

NOM 

Hi 

0.04876926 

SEPM 

HI 

50-100K 

VHl 

VHI 

0.18911487 

GOVT  vs  NON-GOVT 

HI 

<50K 

HI 

NOM 

0.10334420 

TOOLING 

HI 

<50K 

HI 

NOM 

0.10334420 

RDT&E 

VHI 

50-100K 

HI 

HI 

0.16294149 

SURVIVABILITY 

HI 

50-100K 

HI 

HI 

0.11191567 

TEST  COSTS 

VHl 

100-150K 

HI 

HI 

0.18748558 

DEW  PROTO  TO  PROD 

NOM 

100-150K 

NOM 

HI 

0.07331334 

MADCAP 

HI 

SO-IOOK 

HI 

HI 

0.11191567 

MISSILE  ATTITUDE  CNTL 

VHI 

50-100K 

VHI 

NOM 

0.20323474 

ENVIRONMENTAL  I 

VHI 

50-100K 

VHI 

VHI 

0.24014069 

ENVIRONMENTAL  11 

VHl 

50-100K 

VHl 

VHl 

0.24014069 

ENVIRONMENTAL  III 

VHI 

100-ISQK 

VHI 

VHI 

0.26468478 

O&S  CERs 

VHI 

50-I00K 

VHI 

HI 

0.21708095 

PROTO-PROD  FACTORS 

VHI 

100-I50K 

VHI 

HI 

0.24162503 
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TABLE  3-1 

PROJECT  RELATIVE  VALUES  (Cont’d) 


MODELS 

REACT  ENHANCEMENTS 
STACM  ENHANCEMENTS 
ACEITLBRARIES 
GEP  ENGRG.  &  COST 
ST&E  LCC  TRADES 
EP  ENGRG.  &  COST 
CCE  ENGRG.  &  COST 
SCATS 

PICES  SUPPORT 
ACEIT  SUPPORT 
GUARDIAN  MODEL 
DEFENDER  MODEL 
SENSOR-INTERCEPTOR 
COST/PERFORMANCE 
TRADE-OFF  MODEL 
ENVIRONMENTAL  11 
ENVIRONMENTAL  III 
MODIBLE  RADAR  ICE 
ACEIT  TO  BIG  3  FORMAT 
GENERIC  INT.  MODEL 


SPECIAL  STUDIES 
S/W  SIZING  RISK  ANAL 
SAR  COST  GROWTH 
BM/C.,  SIZING 
MICROELEX  SIZING 
ANALOG  SELECTION 
SAV  FUNCT  PT  ANALYSIS 
AI/EXPERT  SYSTEMS 
LINEAR  DISCRIMINATORS 
ARMY  REVIEW  PROCESS 
LEAP  ARCH.  ANALYSIS 
WBS  &  CCDR  PLANNING 
ARMY  SPACE  APPLICATN 
SMALL  PRGM  COST  RES 
LIQUID/GEL  PROPELLANT 


URGENCY 

0.0559728 

AFFORD 

0.0665298 

NOM 

<50K 

NOM 

<50K 

VHI 

50-100K 

VHI 

I00-150K 

HI 

I00-I50K 

VHI 

100-150K 

VHI 

lOO'lSOK 

LOW 

<50K 

NOM 

SO-IOOK 

VHI 

50-100K 

LOW 

100'150K 

LOW 

I00'150K 

HI 

I50-200K 

VHI 

50-100K 

VHI 

50-100K 

HI 

150-200K 

VHI 

<50K 

HI 

I50-200K 

URGENCY 

AFFORD 

0.02087488 

0.02481208 

VHI 

<50K 

HI 

<50K 

NOM 

<50K 

NOM 

50-I00K 

NOM 

<50K 

HI 

50-I00K 

LOW 

<50K 

LOW 

<50K 

HI 

<50K 

NOM 

50-I00K 

NOM 

<50K 

HI 

I50-200K 

NOM 

50-100K 

NOM 

<50K 

PAYOFF 

0.0593883 

FEASIBLE 

0.0252954 

NOM 

VHI 

NOM 

VHI 

HI 

VHI 

VHI 

HI 

HI 

HI 

VHI 

HI 

VHI 

HI 

NOM 

VHI 

NOM 

VHI 

VHI 

VHI 

LOW 

HI 

NOM 

VHI 

HI 

HI 

VHI 

HI 

VHI 

HI 

HI 

VHI 

VHI 

VHI 

HI 

NOM 

PAYOFF 

FEASIBLE 

0.02214868 

0.00943384 

VHI 

NOM 

NOM 

HI 

NOM 

HI 

NOM 

NOM 

NOM 

VHI 

NOM 

HI 

HI 

LOW 

NOM 

LOW 

HI 

VHI 

HI 

HI 

NOM 

VHI 

VHI 

HI 

HI 

NOM 

HI 

NOM 

RELATIVE 

VALUE 

0.02635481 

0.02468682 

0.06357704 

0.08258980 

0.02778136 

0.02505922 

0.02505922 

0.02468682 

0.02455186 

0.08208244 

0.02162146 

0.03127327 

0.05449380 

0.07420039 

0.07420039 

0.06237584 

0.08388539 

0.04976103 


RELATIVE 

VALUE 

0.02658014 

0.01079505 

0.00688936 

0.00445188 

0.00982894 

0.01012264 

0.00836511 

0.00422109 

0.01787865 

0.01036097 

0.00982894 

0.02722482 

0.00859590 

0.00926831 
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3.4 


The  relative  values  for  each  project  can,  in  a  simplistic  sense,  be  viewed  as 
representing  a  measure  of  "profit"  that  would  accrue  to  CAO  should  the  project  be 
undertaken.  Of  course,  it  is  much  more  since  the  relative  value  represents  the  combination  of 
multiple  criteria,  in  a  quantified  proportion,  integrated  into  one  number  for  each  project. 
These  project  relative  values  can  then  be  utilized  in  an  integer  linear  program  in  the  same 
way  that  profit  is  utilized  as  the  objective  function  of  a  linear  programming  maximization. 
But  now  the  objective  function  represents  many  objectives. 

The  choice  variables  in  the  objective  function  are  constrained  to  be  binary  (0  or  1). 
In  other  words,  a  project  is  chosen  or  it  is  not.  Neither  multiple  copies  nor  fractional  parts  of 
a  project  can  be  chosen  as  "might  occur"  in  a  linear  program.  Another  set  of  constraints  have 
to  do  with  the  total  budget  available  over  any  time  period,  and  how  much  of  it  should  be 
allocated  to  each  major  research  area.  In  our  analysis,  the  total  budget  was  not  a  fixed 
number  but  rather  was  a  range  (e.g.,  $600-800K  for  a  single  fiscal  year).  The  relative  amount 
of  the  total  budget  allocated  to  any  single  research  area  was  treated  as  a  midpoint  percentage 
range  of  the  total  budget  range.  For  example,  the  database  research  area  might  be 
constrained  to  be  between  10%  and  20%  of  a  total  budget  of  $700K  (i.e.,  $70-140K).  Other 
constraints  could  easily  be  added,  but  were  not  for  this  analysis.  In  our  analysis  the  budget 
constraints  were  set  as  follows: 

Total  Budget  $600  -  800K  per  fiscal  year 

Database  at  least  1 5%  of  total  budget  midpoint 

CERs/Methods  40  -  60%  of  total  budget  midpoint 
Models  at  least  25%  of  total  budget  midpoint 

Special  Studies  at  least  5%  of  total  budget  midpoint 

The  What's  Best  spreadsheet  add-in  by  Lindo  Systems,  Inc.  was  used  to  perform  the 
ILP  selection  of  projects.  The  spreadsheet  used  was  Excel  4.0  by  Microsoft  Corporation. 

In  summary,  the  multicriteria  approach  consists  of  using  AHP  to  derive  measures  of 
value  for  the  proposed  research  projects.  This  allows  many  objectives  to  be  combined  into  a 
composite  measure  of  values  to  be  maximized.  Then  ILP  is  used  to  chose  the  best  set  of 
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projects  subject  to  a  set  of  budget  constraints.  Since  the  demands  on  CAO  are  not  static,  this 
process  can  be  repeated  as  of^en  as  necessary,  changing  judgments  in  the  AHP  model  or 
adding  new  constraints  whenever  external  events  make  them  appropriate.  Since  the  analysis 
utilizes  commercial  software,  the  files  associated  with  the  analysis  can  be  saved  and,  thus,  are 
available  to  update  the  project  choice  list  on  an  as  needed  basis. 


3 


24 


4 

RESEARCH  ROADMAP 

Projects  were  selected  using  the  methodology  described  in  Section  3.  The 
selections  were  subdivided  into  near-term  and  far-term  projects.  Near-term  was  defined  as 
the  next  two  fiscal  years,  and  far-term  was  years  three  through  five.  Time-phasing  was 
accomplished  by  setting  fiscal  year  budget  thresholds  within  the  ILP.  In  particular,  the  next 
fiscal  year's  budget  was  constrained  to  be  between  $600K  and  $800K.  The  second  fiscal 
year's  budget  was  constrained  to  $1200'1600K.  The  full  five  year  budget  was  constrained 
between  $3000-$4000K.  Tables  4-1, 4-2,  and  4-3  show  the  scheduling  of  near-term  and  far- 
term  projects. 


Table  4-1 

Near-Term  Projects  -  First  Fiscal  Year 


Databases 

Estimated  Cost 

ASAT 

SO-IOOK 

ERIS  (GBI) 

50-100K 

CERs/Methods 

TELESCOPE 

<50K 

DEWAR/CRYOSTAT 

<50K 

WINDOW  COOLING 

<50K 

ENVIRONMENTAL  I 

50-100K 

Models 

REACT  Enhancements 

<50K 

STACM  Enhancements 

<50K 

ACEIT  SUPPORT 

50-100K 

ACEIT  TO  BIG  3  FORMAT 

<50K 
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Table  4-2 

Near-Term  Projects  -  Second  Fiscal  Year 


Databases 

Estimated  Cost 

ERIS 

100-150K 

CERs/Methods 

SEPM 

ADVANCED  SOLID  PROPELLANTS 

MATERIAL  COST  IMPACTS 

WARHEADS/LEDS 

ENVIRONMENTAL 

Models 

ACEIT  Libraries 

50-100K 

SCATS 

<50K 

Special  Studies 

WBS  AND  CCDR  PLANNING 

<50K 
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Table  4-3 

Far-Term  Projects  -  Fiscal  Years  Three  -  Five 


Databases 

Estimated  Cost 

Aegis 

50-1 OOK 

Hazardous  Waste  Handbook 

50-1 OOK 

CERs/Methods 

Signal  Processor 

50-1  OOK 

Detector  Arrays 

50-100K 

Visible  Seekers 

50-100K 

Pointing,  Tracking,  Retargeting 

<50K 

Investment  Items 

50-1  OOK 

Integration  Risk  Factors 

<50K 

Govt  Vs  Non-Govt 

<50K 

Tooling 

<50K 

Rdt&E 

50-1  OOK 

Missile  Attitude  Control 

50-1  OOK 

Environmental 

50-1  OOK 

O&S  Cers 

50-1  OOK 

Prototype-To-Production  Factors 

100-1 50K 

Hypervelocity  Gun  Technology 

<50K 

Models 

GEP  Engineering  &  Cost 

100-1 50K 

PICES  Support 

50-1  OOK 

Enviroiunental 

50-1  OOK 

Special  Studies 

■■HBI 

SAV  Sizing  Risk  Analysis 

SAR  Cost  Growth 

Liquid/Gel  Propellants 

<50K 
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The  suggested  lists  and  evaluation  techniques  are  presented  as  an  approach  to  the 
CAO's  long-range  planning.  Tecolote's  objective  was  to  present  the  USASSDC's 
requirements,  priorities,  and  budget  constraints  as  recognized  by  us  during  this  task.  The 
next  step  would  require  feedback  from  surveys  or  interviews  with  key  USASSDC  CAO 
personnel.  That  information  will  form  the  foundation  for  calibrating  the  "real  plan"  based  on 
a  consensus  (using  AHP)  and  sorting  (using  ILP)  of  CAO’s  actual  priorities.  This  would 
initiate  an  efficient  process  for  updates  as  priorities  change. 

This  decision  process  presented  herein  is  a  unique  application  of  operations  research 
decision  theory.  This  multi-criteria  objective  function,  using  AHP  for  setting  coefficients  and 
applying  constraints  with  linear  programming,  has  been  shown  to  be  theoretically  valid.  We 
are  not  aware  of  previous  practical  applications  It  should  be  noted  that  this  process  could  be 
tailored  to  many  decision-making  tradeoffs  that  USASSDC  faces.  Therefore,  a  spin-off  of 
this  task  could  be  a  useful  tool  for  program  management  problems  that  transcend  cost 
research  planning. 
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